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Introduction 


The  objective  of  this  project  is  to  validate  a  combined  EEG  and  eye  tracking  system  aimed  at  assessing 
compromised  cognitive  function  stemming  from  mild  traumatic  brain  injury  (mild  TBI).  Research 
suggests  that  the  neural  injuries  resulting  from  mild  TBI  do  not  always  produce  observable  performance 
deficits.  However,  subjective  ratings  suggest  that  the  level  of  effort  required  to  perform  at  a  given  level 
can  be  higher  with  mild  TBI;  associated  neuroimaging  data  reveal  a  broader  recruitment  of  cortical 
neurons  to  accomplish  tasks  in  mild  TBI  relative  to  uninjured  individuals  [1-3].  The  research  described 
here  combines  information  from  two  distinct  physiological  sensing  approaches  to  make  inferences 
about  injury-related  changes  in  cognitive  function  using  measures  that  are  sensitive  to  cognitive  effort. 
The  goal  is  to  combine  the  expertise  of  academic,  military,  and  industry  researchers  to  create  a 
practical  and  effective  neurodiagnostic  assessment  tool  that  can  be  used  in  a  broad  range  of  contexts 
in  which  cognitive  assessment  is  relevant.  Validation  of  the  integrated  EEG  and  eye  tracking  system 
will  include  evaluation  of  the  specificity  and  sensitivity  of  these  measures  based  on  characterizations  of 
injury  severity,  performance  on  a  neurocognitive  test  battery,  and  self-report  measures  of  cognitive 
efficacy.  We  will  also  include  functional  magnetic  resonance  imagining  (fMRI)  and  diffusion  tensor 
imaging  (DTI)  to  characterize  the  extent  of  functional  cortical  recruitment  and  white  matter  injury, 
respectively.  The  inclusion  of  fMRI  and  DTI  will  provide  an  objective  basis  for  cross-validating  the  EEG 
and  eye  tracking  system.  Both  the  EEG  and  eye  tracking  data  will  be  collected  in  the  context  of  a  dual¬ 
task  experimental  paradigm  with  visual  target  detection. 

Keywords 

TBI,  concussion,  saccadic,  eye  tracking,  EEG,  cognition 

Overall  Project  Summary 

During  the  Year  2  period  of  performance  (03/08/2014  -  03/07/2015),  significant  progress  was  made  in 
the  pilot  study  and  cognitive  task  development.  Key  milestones  are  approximately  4  quarters  behind 
schedule  due  to  delays  in  initial  contract/subaward  negotiations  and  transition  to  a  new  laboratory 
location.  The  laboratory  transition  was  conducted  in  order  to  mitigate  the  negative  impact  of  changes  to 
base  access  policies  implemented  at  the  Naval  Support  Activity  Bethesda  (NSAB),  resulting  in  a  longer, 
more  difficult  process  to  acquire  base  access  for  many  participants.  We  will  attempt  to  make  up  time 
through  accelerated  recruitment  and  data  collection  efforts.  The  Fusion  primary  study  has  been 
approved  by  USU  IRB  #1  and  HRPO;  recruitment  efforts  and  data  collection  will  commence  in  April 
2015.  Please  note  that  carry-forward  funds  will  be  used  to  complete  tasks  initially  scheduled  for  Year  2 
but  now  scheduled  for  Year  3. 


Progress  related  to  specific  milestones  as  outlined  in  the  statement  of  work  is  outlined  below: 

Hire  and  train  research  personnel.  Personnel,  including  a  Research  Scientist  and  a  Research 
Assistant  I  were  hired.  Training  for  primary  study  procedures  including  neuropsychological  testing,  eye¬ 
tracking,  virtual  reality  driving  simulator,  and  EEG  data  acquisition  is  ongoing  for  all  research  personnel. 

Obtain  IRB  approvals.  USU  IRB  #1  approval  for  the  primary  study  was  received  on  January  12,  2015. 
HRPO  approval  for  the  primary  study  was  received  March  27,  2015. 

Develop  cognitive  efficacy  tasks  for  VR,  PC,  and  MRI  environments.  Two  PC-based  Fusion  tasks 
(N-back  and  Color  Match)  and  one  VR-based  Fusion  task  (Coastal  Route)  have  been  developed, 
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tested,  and  revised  across  multiple  iterations  to  the  current  final  versions  for  the  primary  study.  The 
fMRI-based  Fusion  task  has  been  developed  and  will  be  tested  in  the  near  future. 

The  goal  of  the  Fusion  tasks  is  to  evaluate  changes  in  behavioral  performance,  saccadic  performance, 
and  brain  activation  that  occur  at  each  level  of  cognitive  load.  It  is  expected  that  individuals  with  poorer 
cognitive  efficacy  resulting  from  TBI  will  demonstrate  more  significant  performance  trade-offs  and 
greater  increases  in  brain  activation  with  increasing  load. 

Fusion  N-Back  Task. 

The  Fusion  N-back  task  is  a  target  detection  task  that  features  three  conditions  requiring  varying  levels 
of  cognitive  workload.  During  the  simple  reaction  time  condition  (SRT;  low  cognitive  workload), 
participants  must  fixate  on  a  cross  in  the  center  of  the  screen.  After  a  period  of  time,  the  cross  is 
replaced  by  one  of  three  cues:  directional  arrow  (DC),  mis-directional  arrow  (MDC),  or  no  cue  (NC;  the 
cross  persists).  After  200ms,  a  target  (white  circle)  appears  on  the  left  or  the  right.  Participants  are 
instructed  to  shift  their  gaze  to  the  target  and  press  a  button  on  the  response  box  as  quickly  as 
possible.  The  choice  reaction  time  condition  (CRT;  moderate  cognitive  workload)  is  similar  to  the  SRT 
condition,  except  the  targets  appear  on  the  left  or  right  as  blue  or  green  circles.  Participants  must  gaze 
at  the  target  and  press  one  button  if  the  target  is  green  and  a  different  button  if  the  target  is  blue.  The  1  - 
back  condition  (high  cognitive  workload)  requires  participants  to  remember  the  color  of  the  last  target 
that  appeared  (blue  or  green).  When  the  next  target  appears,  participants  must  gaze  at  the  target  and 
press  a  button  based  on  the  color  of  the  previous  target. 

Fusion  Color  Match  Task. 

The  Fusion  Color  Match  task  is  a  dual  task  that  requires  participants  to  perform  a  vigilance  task  and  a 
target  detection  task  simultaneously.  Participants  see  a  square  in  the  center  of  the  screen.  When  the 
task  begins,  the  border  of  the  square  changes  color  and  the  participant  must  continually  try  to  match 
the  color  of  the  border  to  the  color  of  the  inside  of  the  square  by  pressing  a  button.  While  completing 
this  task,  the  participant  must  also  fixate  on  a  cross  that  is  overlaid  on  the  center  of  the  square.  After  a 
period  of  time,  this  cross  is  replaced  by  one  of  three  cues  (DC,  MDC,  or  NC).  After  200ms,  a  target 
(white  circle)  appears  on  the  left  or  the  right.  Participants  are  instructed  to  temporarily  shift  their  gaze 
to  the  target  while  continuing  to  adjust  the  color  of  the  border  of  the  square.  Cognitive  workload  is 
manipulated  by  adjusting  the  rate  at  which  the  color  of  the  border  changes  (slow  =  low  cognitive 
workload,  medium  =  moderate  cognitive  workload,  fast  =  high  cognitive  workload). 

Fusion  Coastal  Drive 

The  Fusion  Coastal  Drive  takes  place  in  the  virtual  reality  driving  simulator.  Participants  are  instructed 
to  drive  along  a  coastal  highway  while  performing  the  target  detection  task.  Participants  must  fixate  on 
a  cross  in  the  center  of  the  screen  which  is  replaced  by  one  of  three  cues  (DC,  MDC,  or  NC).  After 
200ms,  a  target  (white  circle  with  a  number,  0-9,  in  the  center)  appears  on  the  left  or  the  right. 
Participants  are  instructed  to  shift  their  gaze  to  the  target  and  say  the  number  in  the  target  circle  aloud 
to  the  examiner  while  trying  to  maintain  lane  position.  Motor  tracking  (steering)  difficulty  is  modulated  by 
the  speed  of  the  vehicle. 

Fusion  fMRI  Task 

The  fMRI-CET  is  a  target  detection  task  that  features  two  conditions  requiring  varying  levels  of 
cognitive  workload.  This  is  an  fMRI  analog  of  the  PC-based  Fusion  N-back  task.  During  the  choice 
reaction  time  condition  (CRT),  participants  must  fixate  on  a  cross  in  the  center  of  the  screen.  After  a 
period  of  time,  the  cross  is  replaced  by  one  of  three  cues  (DC,  MDC,  or  NC).  After  200ms,  a  target 
(white  circle  with  a  ‘6’  or  ‘9’  in  the  center)  appears  on  the  left  or  the  right.  Participants  are  instructed  to 
shift  their  gaze  to  the  target  and  press  one  button  if  the  target  is  a  ‘6’  and  a  different  button  if  the  target 
is  a  ‘9.’  The  1-back  condition  requires  participants  to  remember  the  number  of  the  last  target  that 
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appeared  (‘6’  or  ‘9’).  When  the  next  target  appears,  participants  must  gaze  at  the  target  and  press  one 
button  if  the  number  of  the  current  target  is  the  same  as  the  previous  target  and  a  different  button  if  the 
number  of  the  current  target  is  different  than  the  previous  target. 


Pilot  test  each  of  the  cognitive  efficacy  tasks  among  30  healthy  control  participants.  Data 
collection  for  the  pilot  study  is  almost  completed  (n  =  29  of  30).  Preliminary  analyses  of  performance 
and  self-report  data  suggest  that  the  cognitive  efficacy  tasks  successfully  discriminate  between 
workload  conditions  and  cue  types. 

For  the  Fusion  N-Back  Task,  Graph  1  illustrates  significant  main  effects  of  load  (F  (2,  30)  =  8.51 ,  p  < 

.05,  partial  eta  squared  =  .362)  and  cue  type  (F  (2,  30)  =  19.98,  p  <  .05,  partial  eta  squared  =  .571)  on 
saccadic  reaction  time.  Graph  2  illustrates  significant  main  effect  of  load  (F  (2,  28)  =  21.98,  p  <  .05, 
partial  eta  squared  =  .611)  and  cue  type  (F  (2,  28)  =  17.412,  p  <  .05,  partial  eta  squared  =  .554)  on 
manual  reaction  time. 

For  the  Fusion  Color  Match  Task,  Graph  3  illustrates  significant  main  effects  of  load  (F  (2,  32)  =  19.236, 
p  <  .05,  partial  eta  squared  =  .546)  and  cue  type  (F  (2,  32)  =  22.807,  p  <  .05,  partial  eta  squared  = 

.588)  on  saccadic  reaction  time.  Graph  4  illustrates  a  significant  main  effect  of  load  (F  (2,  46)  =  17.269, 
p  <  .05,  partial  eta  squared  =  .429)  on  performance  (average  difference  between  RGB  value  of  outer 
square  versus  the  inner  square). 

For  the  Fusion  Coastal  Drive  Task,  Graph  5  illustrates  a  significant  main  effect  of  cue  type  (F  (2,8)  = 
7.503,  p  <  .05,  partial  eta  squared  =  .652)  on  saccadic  reaction  time.  Graphs  6  and  7  illustrate  a 
significant  main  effect  of  load  on  average  lane  position  (F  (2,20)  =  68.559,  p  <  .05,  partial  eta  squared  = 
.873)  and  average  swerving  (F  (2,20)  =  221 .901 ,  p  <  .05,  partial  eta  squared  =  .957). 

Preliminary  ERP  analyses  were  completed  for  the  Fusion  Color  Match  Task  and  indicate  that  there 
appears  to  be  an  effect  of  condition  and  cue  type  in  neural  responses.  Graphs  8  and  9  illustrate 
significant  main  effects  of  condition  (F(2,34)  =  3.800,  p  <  .05)  and  cue  type  (F(2,34)  =  1 1 .78,  p  <  .05)  on 
frontal-central  activation  250  -  350ms  post  target  onset.  Graphs  10  and  1 1  illustrate  a  significant  main 
effect  of  cue  type  (F(2,34)  =  12.967,  p  <  .05)  on  central  parietal  activation  90  -  175ms  post  target 
onset.  There  is  a  trend  for  a  main  effect  of  condition  on  occipital  activation  100  -  150ms  post  cue  onset 
(see  Graphs  12  and  13).  Additional  ERP  analyses  will  be  completed  following  refinements  to  the 
analysis  pipeline. 

Disseminate  project  plans  and  progress. 

See  Publications,  Abstracts,  and  Presentations  below  for  information  about  progress  in  disseminating 
project  plans  and  progress. 

Update  DOD/TATRC  representatives  on  progress  at  annual  meeting.  Dr.  Ettenhofer  attended  the 
Military  Operational  Medicine  Research  Program  (MOMRP)  meeting  at  Fort  Detrick,  MD  from  July  21  - 
July  22,  2014.  Progress  updates  were  provided  to  DOD/TATRC  representatives  and  revisions  to  the 
protocol  were  implemented  based  upon  feedback  received  at  this  meeting. 
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Key  Research  Accomplishments 

•  Developed  a  novel  task  and  methods  to  effectively  discriminate  cognitive  workload  using  eye 
movements  and  EEG 

•  Algorithms  are  in  development  for  integrated  EEG  and  eye  tracking  analyses 


Conclusions 

The  primary  goal  of  the  Fusion  project  is  to  develop  and  validate  methods  for  evaluating  changes  in 
behavioral  performance,  saccadic  performance,  and  brain  activation  that  occur  at  different  levels  of 
cognitive  load.  It  is  expected  that  individuals  with  poorer  cognitive  efficacy  resulting  from  TBI  will 
demonstrate  more  significant  performance  trade-offs  and  greater  increases  in  brain  activation  with 
increasing  load.  Preliminary  analyses  suggest  that  the  Fusion  tasks  that  will  be  used  for  integrated  eye 
tracking  and  neural  monitoring  are  able  to  successfully  discriminate  cognitive  workload  across  difficulty 
levels.  This  provides  a  promising  basis  for  the  extraction  of  valuable  data  relevant  to  detecting  cognitive 
efficacy  after  TBI.  Ongoing  research  will  evaluate  the  Fusion  tasks  for  this  purpose,  and  examine 
potential  mechanism  of  any  effects  observed. 

Publications,  Abstracts,  and  Presentations 

The  following  paper,  describing  study  progress  and  plans,  has  been  peer-reviewed  and  accepted 
during  the  project  period: 

Safford,  A.,  Kegel,  J.,  Hershaw,  J.,  Girard,  D.,  &  Ettenhofer,  M.L.  (2015,  Aug).  Eye-movement  Based 
Detection  of  Subtle  Cognitive  Impairment  for  Enhanced  Assessment  of  Mild  Traumatic  Brain  Injury. 
Paper  to  be  presented  at  the  1 7th  Annual  Conference  on  Pluman-Computer  Interaction,  Los  Angeles, 
CA.  Conference  proceedings  published  in  Lecture  Notes  in  Computer  Science. 

The  following  invited  presentations,  given  by  Dr.  Ettenhofer  during  the  project  period,  have  included 
dissemination  of  scientific  methods  and  results  of  this  study: 

Ettenhofer,  M.L.  (March,  2015).  “Novel  Technologies  for  Assessment  and  Rehabilitation  of  Traumatic 
Brain  Injury.”  Brain  Injury  Awareness  Seminar,  Walter  Reed  National  Military  Medical  Center. 

Ettenhofer,  M.L.  (January,  2015).  “Novel  Technologies  for  Assessment  and  Rehabilitation  of  Traumatic 
Brain  Injury.”  Annual  Council  of  Directors  Meeting,  Henry  M.  Jackson  Foundation  for  the  Advancement 
of  Military  Medicine. 

Ettenhofer,  M.L.  (September,  2014).  “The  Eyes  as  Windows  to  the  Brain:  A  Saccadic  Approach  to 
Neuropsychological  Assessment.”  Neuro-lnfectious  Disease  Seminar  Series,  National  Institute  of 
Neurological  Diseases  and  Stroke,  National  Institutes  of  Health. 

Ettenhofer,  M.L.  (June,  2014).  “Eliciting  and  Measuring  Neurocognitive  Processes  Using  Eye 
Movements.”  Clinical  Center,  National  Institutes  of  Health,  Bethesda,  M.D. 

Ettenhofer,  M.L.  (April,  2014).  “Neurocognitive  Eye  Tracking:  Rationale,  Methods,  and  Applications.” 
War  Related  Injury  and  Illness  Study  Center  (WRIISC),  Veterans  Affairs  Medical  Center,  Washington, 
DC. 
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Inventions,  Patents,  and  Licenses 


None  to  report. 

Reportable  Outcomes 

Nothing  to  report. 

Other  Achievements 

Nothing  to  report. 
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Graph  2 


N-Back  CET  -  Manual  RTs 
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Graph  3 


Color  Match  CET  -  Saccadic  RTs 
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Average  Difference  from  Center  Color  (RGB  value) 


Graph  4 

Color  Match  CET  -  Color  Match  Performance 
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Graph  5 


Fusion  Coastal  Drive:  Saccadic  RTs 
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Average  Distance  from  Center  of  Lane 


Graph  6 


Fusion  Coastal  Drive:  Average  Lane  Position 


Graph  7 


Fusion  Coastal  Drive:  Average  Swerving 


Graph  8 


Frontal-Central  Activation  post  Target 
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Amplitude  (pV) 


Graph  12 


Occipital  Activation 


3 


2 


1 


0 


-1 


-2 


Time  (ms) 


-  Low  Directional  Cue 

-  Low  Misdirectional  Cue 

-  Moderate  Directional  Cue 

Moderate  Misdirectional  Cue 

-  High  Directional  Cue 

—  High  Misdirectional  Cue 


22 


Amplitude  (pV) 
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